Repository navigation
fix: update kernel pin and keep the close-race error typed - #619
Merged
Merged
Conversation
Pin the kernel to dac32e2 on main. It includes databricks/databricks-sql-kernel#462: AsyncStatement.close() now releases its canceller and the result streams of its handles, so a closed async statement no longer keeps the session, and the host's shared telemetry client, alive until GC. It also picks up #451, #473 and #454. The regenerated native/kernel/index.d.ts only adds #462's doc comments. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Nov 16, 2026
…rror
With the new kernel pin, AsyncStatement.close() empties the result handle,
so a fetchChunk() or hasMore() in flight on SEA rejects with the kernel's
InvalidStatementHandle ("AsyncResultHandle already closed"), surfaced as a
generic HiveDriverError. Check the lifecycle first in their catch blocks so
a local close or cancel still reports OperationStateError(Closed/Canceled).
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Medium · 1 Low
Mostly good: the pin bump and the typed close-race error are sound. One medium regression: the new guard also catches local cancels, so a cancel during an in-flight fetch no longer closes the kernel statement, and the facade never closes it after a cancel.
The guard in the fetchChunk() and hasMore() catch blocks threw before fetchFailure() ran, so a fetch interrupted by cancel() skipped the cleanup close. After a cancel, nothing else closes the statement, so it and its session stayed alive until GC. Move the check into fetchFailure(): record whether a local close or cancel happened, run the cleanup close, then report the lifecycle state. The flags are read first because the cleanup close sets isClosed. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
With the new kernel pin, AsyncStatement.close() empties the result handle even when its close RPC fails, but kernelClose rolls isClosed back. A fetch that then hits the emptied handle reported the kernel's HiveDriverError "AsyncResultHandle already closed" instead of OperationStateError(Closed). On the old pin it kept returning rows. Record the close() request in a flag that the failure doesn't reset and treat it in fetchFailure() like a local close. Cleanup closes don't set it. fetchFailure() now builds the Canceled/Closed error itself, so the result doesn't depend on its own cleanup close succeeding. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Low
Looks good, with one low concern: a concurrent fetch can report OperationStateError(Closed) instead of the real error, because the cleanup close sets isClosed too. The closeRequested flag, the cleanup-close ordering and the new unit tests cover the close, cancel and failed-close races as the PR describes.
fetchFailure() counted isClosed as a local close, but its own cleanup close sets that flag before its first await. When two fetches failed on the same error, the second reported OperationStateError(Closed) instead of the error. closeRequested now covers every client close(), so check only it and isCancelled. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
This was referenced Nov 17, 2026
Open
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates
KERNEL_REVfrom0b5d0edtodac32e2, the current kernelmain. The pin picks up databricks/databricks-sql-kernel#462:AsyncStatement.close()now releases its canceller and the result streams of theAsyncResultHandles it returned. Before, a closed async statement kept the kernel session, and the host's shared telemetry client, alive until JS garbage collection. A later session on the same host then reused that client and ignored its owntelemetryBatchSize/telemetryFlushIntervalMs. This fixes TELEMETRY-015 on SEA, which databricks/databricks-driver-test#2069 tracks as an xfail tripwire.The pin also picks up databricks/databricks-sql-kernel#451 (reuse a successful async terminal response instead of re-polling), #473 (OSV dependency fixes) and #454 (drop presigned query strings from request errors and debug URLs). The old pin
0b5d0edwas the #436 branch head; its tree matches #436's merge on main (db23442). The regeneratednative/kernel/index.d.tsonly adds #462's doc comments onAsyncStatement.close()andAsyncResultHandle.fetchNextBatch().Close during an in-flight fetch. Because #462 empties the result handle on close, an SEA operation closed while a multi-batch fetch is in flight made that fetch reject with the kernel's
InvalidStatementHandle, surfaced as a genericHiveDriverError: AsyncResultHandle already closed. On the old pin it rejected withOperationStateError(CLOSED).fetchFailure(), which handles a failedfetchChunk()orhasMore(), now records whether a local close or cancel happened, runs its usual cleanup close, and then reportsOperationStateError(Closed/Canceled)instead of the decoded kernel error. Keeping the cleanup close matters after a cancel: nothing else closes the statement then. The fetch still stops early, as #462 intends. #462 also empties the handle when the close RPC fails, whilekernelCloserollsisClosedback so the close can be retried. Soclose()now sets acloseRequestedflag that a failure doesn't clear (cleanup closes don't set it), andfetchFailure()treats it as a local close. A fetch hit by a failed close now getsOperationStateError(CLOSED)instead of theHiveDriverError; on the old pin it kept returning rows.Validation, with the native binding built from the new pin:
lint,prettier --check,type-check, and 1,785 unit tests. The newKernelOperationBackend — close()/cancel() during an in-flight fetchtests (fetchChunk()andhasMore()) assert the typed error and a singlestatement.close(), and cover aclose()whose RPC fails (retry resolving, as in the kernel, or rejecting), which takes one cleanup close. Two concurrent fetches that fail on the same error both report that error, notCLOSED. Without the change the close and failed-close cases fail with theHiveDriverError.0b5d0edthe nextfetchNextBatch()still returns a batch; ondac32e2it rejects withInvalidStatementHandle. A secondclose()resolves on both.SELECT id, uuid() FROM range(0, 3000000), fetch 1,000 rows, startfetchChunk({ maxRows: 3e6 }), interrupt after 1 s):close():OperationStateError CLOSEDin 3/3 runs, settling in about 1.07 s (4/4 again with the failed-close change, about 1.09 s). Without the change, 3/3 runs giveHiveDriverError "AsyncResultHandle already closed".cancel()with CloudFetch off (paged results, so the next page fails):OperationStateError CANCELEDin 3/3 runs (4/4 again with the failed-close change), and the kernel statement is closed.main(e40743249):TELEMETRY-0*), serial, under--max-semi-space-size=256: 14 passing, including TELEMETRY-015.XPASSin 2/2 runs. That PR flips the cell toimplementedbefore it merges.Merge prerequisite: databricks/databricks-driver-test#2146 re-records seven Node SEA replay recordings for kernel#451's dropped re-poll, and must merge before this PR enters the merge queue.
This pull request and its description were written by Isaac.